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The present study sought^to discover the effects on 
delayed reten*tion of two different ^egrees bfc[uality of initial 
instruction for two Afferent amounts of learning time; which 
combina*iiion of quality and time is most likely 'to be achieved in 
normal school situations; an^ the implications for ^hose who design 
instructional materials!. T,w6 programed texts, one ^superior to the 
ot^er in the -quality of produced level of Inditial learning, were 
pjyepared 'and administered to 12Qi* high, school seniors randomly , 
assigned to one of four equal -sizedrtreatmen^ groups, in combination 
wi\fi jsach *of two differing amount^ of learning time. An immediate 
posttest and a delayed , retention t,est were administered^, Parametric 
and honparametric statistical tests yielded the following results: 
(1) an ^increased amount' of study time promoted' differ in^ Tevels of 
achievement inJ. delayed retention; (2) existing differences among ' 
groups, could be attributed to effects of diffei^ing 'gueClity, of 
instxiictional levels and/or amount of initial learning} and (3) hp 
interaction effects wjer^ significant. Limitations of the study are 
delineated. (Author/^E) ^c^s. 
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INTRODUCTION 



0'^e*rview 



Sizeable amou'nts of resT5urces\ Ar;,e necessary to maintain an ex- 
tensive 4)ublio education system. ^Support for these exp^enditur es is 
forthcoming, in part, because the system assumes long term retention 
of skills as a goal. Sucfli support would be les.s likely if onLy short 
term i^etent^Lon resulted from the system, as this would preclude long 
term benefits for tjie society which supports education. Therefore, 
an important ipart of the justification for the extensive educational 
endeavor rests upon -the system's capability for ^promoting recall and 
use of skills, concepts, and principles, by students, over extended 
periods of time. T . J 

' . * ■ 

ftow would an edu'cational system promote this capability for long- 
term i:,ecail of essential skills in Its instructional efforts? While, 
varipus possibilities for accomplishing this exist,, three are at once 
apparent: a) provide for instruction which promotes highly effective 
initial learning on the parj: of students served by the syste'm, b) 
allow more time fot- studients to engage in the learning of these J 
essential skills, and c) provide^ for strategically placed review and 
rehearsal periods f ollowing. initia^l instruction in essential skills. 
Of these al t er na t^ives , tv/o (a and b) were selected for investigation 
in this study, v \ 

, The study reported here was coitcerned wij^h the investigation of 
ret*ention of intellectual skills. The experimental treatments pro- 
vided instruction wHich prompted two different amounts of learning 
time. Formative evaluation technique^ were used to develop the in- 
struction to provide differing, leve.ls of initial learning. The con- 
tent of the experimental instruction was concerned with developing'; 
skills, in students which would enable them. to app^-y a m^elhod,. devel- 
oped by toulmin (1958), in analyzing "^^positions taken .o\ social issues.' 
In terms of goals of instruction in social studies, skills needed to".»^ 
analyze positions tafcen on docial i^suei^ are assigned high priority 
by Allen and^Adair (19^1) ,. Fenton (1567),^ and^Masslalas apnd Smith 
'(1965), ae well as other aiithbrities in th-e^fieti. 

Programmed texts were used as the 'delivery system for the ex- 
,perimental instruction to ayoid uncontrolled ^teacher-related influ- 
ences* ' , 

Problem Statement ^ 

The problem investigated here concerned two issues which* are 
directly relevant to^ planning^ des^igning, developing, and implementing 
instruction which wil^ iasur^ retention. The two issues of concern 



2 



are closely interrelated. As was stated earlier, 
apparent possibilities for proniptdng retention of 
a) increase the ef f ec^ iveness of initial learning 
quality of instruction and b) allow s tudents '*mor e 
instruction .is offered. 



two immedia tely 
instruction are to 
by impf oving the 
tame for whatever . 



In impleriienta tion of instruction, insyriug retention wou-ld be 
more likely if .students increased initial learn^ing, i.e. if they had 
access tfo instruction which promoted effectiveness in initial learning. 
Al$o, if students were allowed to participate in the instr^uction for' 
periods ^f time sufficient to increase initin-l Learning; it would 
'seem that retention would be likely to increase. . In light of these 
two** hypo theses , it \rould seem appropr ia.te in the im.plemen ta t*ion of 
instruction, to employ instructio^n that insured a high level of ini-, * 
tial learning effectiveness and/o-r to provide sufficient time for the 
student to engage in instruction to increase effectiveness of initial ^• 
learning. The q.uestion remains \s to which of these two avenues of 
improvement is most practical, and how the two might int.eract. 

So the second is.sue is, "How do' we improve instruction to ful- 
fill heeds impJLied by the two postulates?" To address this question, 
s-ev^ral others must first be faced. First, should ins truction^have 
associated with it, prior to implementation, a Renown le-^el 4)f effect- 
iveness for i^n^uring initial learning? ^^ecdnd. In the realities of 
most instructional environments, is it ^i?^easible to determines effect*- 
iveness of instruction prior to implemejntatio.n? Third, if this latter 
practice is feasible, how effective should instruction be before it 
is used? Finally, if instruction proves to be of a lowc^^r level of 
effectiveness than is desired, hox7 can it ^be feasibly improved, or, 
how can* instructional tWe be utilized 'to compensate for a lower 
level of effectiveness? 

The problem then , revolves around the possibilities of increas- 
ing retention levels of students* as a result of manipulation of a) the 
time students are' engaged in instruction, and/or b)' the level of effect- 
iveness regarding initial learning associated with the instruction 
students engage in* Specifically, the investigation described here 
sought ^to examine the following jaxper imental questions: 

What are the effects upon delayed retentitn of two 
different degrees of quality of initial instruction for 
two different amounts of learning time? Which cbmbina- 
.tion of quality and time is. likely to be most practical 
to achieve in normal school situations'? What ar^ the 
implica t$,ons for those who design instructional mate'ri- 
als? ' ■ ' 
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REI»ATEf) StUDIES, 

The re.sults of o ther . s tudies justify the choice of time -allov/dd 

for instruction and quality of instruction from the Carrpll model as 

variables in the present study of retention of purposeful instruction. 

This is also true of the variable, degree of initial learni^rig, even 

though we 'do no.t define this as Carroll does. 

♦ 

♦ ' * 

Quality of Instruction . " ' ' 

It is diffifcult to locate studies where comprehensive revision 
of instruction for improved quality is considered as an exper imqatal 
variable. According to Sulzen (1972), ^when experimenter^ report the ^ 
use of differing quality levels of instruction in their; studies, they 
most of ten. ad j us t o^nly one of many possible components of instruction, 
e.g., clarify directions, give more examples, usQ more illustrations, 
give objectives to one group and withhold objectives from another," 
etc.^ The lack of studies dealing with entire composites of such com^ 
ponents serves to emphasize the need for such experiments as this . 
one. While iU is profitable tq study, sach single components as those 
just cited, this does not replace the need for varying t;otal effect-^ 
dveness"" of (iomposites. * 

Time Allowed for Instruction . * ** 

Several studies have investigated time allowed for instruction 
as *an independent variable. An experiment by .Deady (L969)^ w,as con-r 
^ cerned with the effects of increased time Allotments on students' 
achievements and atti.tud>&-^irn science classes Students in experi- 
mental groups spent 35 d^ys in a unit; control groups were allowed 
only 20 days. The use of teacher-dependent instruction with several 
teachevrs removed the possibility of controlling for teacher based' 
variance. An analysis of , results from common twists revealed no sig- 
nificant differences between control and experimental grou-ps re^garding 
•either a^t.titude or achievement. 

/ ■ , - 

Corrozi (1970) investigated effects of increased reading time, 
different.typ.es of* ques tions, and faring instructional f-orif^ats orV 
short-term and.dfelayed retention. He founds that reading time h^d a 
positive effect upon both short-term and delayed retention of ijiean- 
xivgful prose material's, but not for honsense|pnaterials . This y^grees 
with Aiisubel * s ^(1970) conclusion that presentation in a meaningful 
context is necessary for the delayed retention of instruction, but 
context exerts no positive effect dn retention of nonsense material. 
* %. 

A post hoc study by Jarvis (1965) found signif i -^-^nt superiority 
f pr increased tiMe .allotments en immediate t^s ts in reading, arithme- 
tic, and language arts achievement, but not on tests of delayed reten- 
tion. This study 'involved 713 sixth graders in 35 schools, so the 
results may have been heavily influenced by lack .of exper iment'al con- 
trol and* by teacher-based variance. 
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Wang and Lindvall (1969),. in periorming a validation study of 
the Carroll model, £ound little Relationship between aptitude and 
learning rate (time taken in study to pass, a mathematics skills test) 
and np significant differences between groups taking short and lengthy 
periods of time in arriving at acceptable performance level? on the 
test. . . » 

In a s tudy involving 2,000 twelfth graders us ing Harvard Project** 
Physics materials, Welch and Bridgham (1968) reported no significant 
difference's , re guarding retention, between g,roups who^ s tudie^the ma-" 
terials for differing time intervals. Teachers selected rnWrvaJ-s 
used> types o-t ins true tignal activities, and ratios of types of in- 
structional activi*tips students were involved in. Therefore, very 
little experimental control v/as exerted in the study. 

Carroll (1973) used aptitude and achievement data gathered from 
standardized testing in a nationv/ide ' ^udy to develop a model ^such 

thajL, a-) for students in any grade, expected achievement is directly 

proportional to measured aptitude, and b) the proportionality coeffi- 
cients (achievement /aptitude) are a function of time, except for some 
cases involving firs.t and second graders. This model demonstrates, 
that achievement at time Jt , ^ the achievement of t,he average individual, 
is the average achievement for aljLt individuals at tliat time. There- 
fore, since the predictor of achieveij^nt from aptitude, is time-corre- 
lated, individuals with slow learning rates (v/ho need more time to 
learn) will have less than a^er-age achievement,, and tKose with rapid 
learning rates will have greater than average achievemei;! t ;«cores. 

To summarize, then, among students concerned v/ith pffec'ts of in- 
creased time allotments, it can be noted that results are mixed. 
Whil,e these mixed results often- seem to result, from lack of standard- ' 
ization of conditions within experiments, it could also be that, the ^ 
increased time intervals .employed were not sufficiently extended to 
account for variance that occurs as a result of including slower learn- 
ers in the experiment ^. * \ j 

Degree of Initial Learning * ^ . 

^ , ' ^ \ . 

In a review of research regarding retention,, Seagoe (1970) proposed 
to teachers t\\dt the single most importaat, factor in preventing forget- 
ting is .degree of initial learning. She. suggested tha,t teachers enhance 
retention by using a) techniques which promote over-learning, b) 
distributed practice during instruction, and c). application of init;ia3 
learning in dlf.fering situation^ following instruction. This empha- 
sis upon at temp ting to promo te the highes t de-gr ee of initial learn- 
ing in order t^o promote retention is in concurrence with guideline^ 
suggested by Gag<nc an4r Rohwer (1969) and Gagne (1970). Similar sug- 
ges^te'd lists of r etentidn-promo ting practices designed to increase 
degree of initial learning have been offered by Davis (1966), who 
adds mental practice (learner rehearsal) to his list, and ^Krumbol tz 
(1964) who suggests the superiority of overt responding over covert 
responding,* » 



' In a study of effects of three types of repetition upon re- a 
tention, W^rine et^ al^ (1972)* found the amo\unt of initial learning tt) 
be a better predictor of retention than any type of repetition in- 
vojLved ift the experiment, tor both specific fact retention and i 
"chynk" retention. This resul.t held for information presented in 
meaningful context and lists of facts presented in i.andom arrange- 
ment« ' ' ^ , . 

Bobrow and Bower (1969) found thfi^t initial learning and reten- 
tion could be increased if students constructed sentences with cri- 
terion material to provide a per soaal ized . bas is for recall. This 
Use of composing of sentences as a method f<^r a construction strategy, 
waich the ^irriters identified as a mathemegenics , is not 'of great 
import* to the present study, but the great differenced in degree of » 
original learning resulting from their use is% ^This difference was . 
reflected in superiority in retent^pn for grou^ps' »with greater initial 
learning. ^ . , ^ 

Several studies have produced results contrary to the findings 
just cited regarding the importance of initial learning in promoting 
.retention, Reynolds and Glaser (1964) found that there no r'e- 

lationship between *the amount of original learning and degree of 
retention, when the differences in original learning resulte-d from 
dtffering numbers of examples used in two versions of '3 programmed 
text in mathematics. Gibson (1969) found that over-learning and 
drill with rules -^Jjromo ted initial learning, but e?certed no signifi- 
cant effect upon delayed retention. Gay (1971), using time for re- 
learning as the measure of retention, found no savin"gs in the number 
of examples necessary in th.e original learning to reach- an arbitrary 
criterion in problem solving using mathematics rules... ^ 
* 

In summary, while the weight of evidence, seems to substantiate 
the iitfportance of ijiitial learning for promoting delayed retent^ion, 
these last three studies reeulting/in contrary findings odealt €«clu-- 
sively with intellectual skills and might be more important in inter- 
prating results of the study described here, as they tend to support 
Gagne's feeling (personal communique) that students either have lUri..- 
tery of intellectual skills or they do not. Tha.t i,s , mas'tety for ^ 
^discrete intellec t.ual skills is likely to be absolute;" it i*3 present^ 
in the learner or it is not. On th^ other hand, jSVer learning of 
information mi y enhance retention* ^ t * *^ 

implications for This Study 



The implications o'f the preceding interpr eta tipns hold importance 
for the design of instructional jnaterials and the assessment of de- 
layed ?:etenfcion^i^n this study due to the following factors. 'First, 
if the conditions which have been identified as promoting retention 
can be designed' into the exp.er imental instruction tO;be used,, the ^ 
possibility of designer bias in favor of any treatment will be less 
likely to occur. Second, if the effect of degree of .initial learning 
upon delayed retention can be validly accounted for b'l^ adjusting re-^ 
tention scores,^ either by using the formula suggested by Davis (19156), 
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or statistically a^s suggested by Wodtka (1967) and Cook (1970)., the 
effeats of amount of time 'allowed' for learning can be more clearly 
isolated, as can the effects of quality of instruction* 



The mixed effects upon retention reporttid here" for increased 
time allotmeats and degree ^of initial learning, coupled with the 
seeming absence of st-udies employing a holistic definition of quality 
Instruction,, especially when quality is not affected by-teacher- , 
dependent activity, serves t^ emphasize the need for studies such as 
the one described here. 



METHOD 



Overviev; 



Th^developmen tal and experimental procedures involved in 'im- 
^plementin^g this study were- performed as follows. First, a programed 
text (referred to in later sections as version I) dealing with* the 
use of Toulmin's (1958) . schema for analyzing position statements was 
developed and implemei^ted with a group o"f 30 high school seniors • 
. Second, data ga ther ed^f rom student comments, test results, and ex- 
perllnenter analyadLs o^ these data, were used to revise the version I 
text yielding a sXcond text (referred to in later sections as version 
II), Third, versioV II was implemented with a second group of 30 
"high school seniors "aVid dc»ta were gathered to insure that version II \ 
was superior to version I in the level of irjitial learning that j 
version II produced for students, < ' ^ 

Fourt^i, \20 high school seniors from two large English classes 
were randomly assigned to one of four equal-sized treatment groups, 

th, each of the two veisions of the programed text was administered 
to one ^of the four treatment grouiJs, in combination with each of 
two diff^erent amounts of learning time. Sixth, when alloted study 
time expired for eac.h group, au immediate pos^test was Administered 
to its, members. Due to len-gth of time requited to complete the 
fifth and sixth proce<iuTes, the 120 studenjts involved reported to 
a large lecture* classroom for 9,0 nyLnutes on each of two cor/secutive 
days. The first day was required for completing the progvrammed 
text and the second was used fi?r completing the programmed text for 
the JLarger study time and for testing*^ Seventh, a ^l^yed .retention 
test* was administered to the 120 students in a single group, 13 
days after the immediate posttest. Finally, data obtained from the 
two tests were subj.ected to parametric (analysis of variance) and 
nonparame tr ic (Wilson Chi-Square) analyses to test ,the effects of 
a) quality of instruction level, b) time engaged in instruction., 
c) level of initial learning^ and d) interactions among -a, b, and^, 
upon delayed reteation of skills in.volved in applying the Toulmin 
schema. 



Expler imentcl * Design 

2 X 2 design with repeated measures was employed in the study 
using two levels of quality^ of instruction and two amounts of study 
time for" each quality of instruction* The two levels of qua*lity.of 
instruction were achieved and documented by implementing two forma- 
tive |eval.uattons . The, first of these documented the quality of the 
first dtaft instructional materials and provided data for revising 
them. The second allowed documentation of the quality level of the 



revised instruction. Two. versions of a* programmed text v;ero emp]iO,yed 
to deliver the ins true tion , each version containing one of the lev'hls 
:of quality whTch resulted from formative eval4iation.. 

An immediate posttest and a. Jd'talayiiJ. ^retention test, comprised the, 
repeated measures factor in the design as each was a iHeasure of re- 
tention' evident at a different po'int in timel In applying parametric 
analysis of j^arianc^- techniques to ddtsc obtained from the experiment, 
the immediate posttest (degree of initial l§a,rning) v/as ^re^ated as 
an independent variable as /su)2gested by Wodtke (/1967). Such a 
practiye''is"^])pro.aria te, aCvj^ording to that write^l^ as measure's of 
indti^^ lear.ti.ingVa^d delayed retention measures are usually highly . 
correlated, ' \?he{r a measure of. delayed retentiort is used as the de- 
pendent;^ var i^blSV^-A^^^^ the case in the present study, consideration 
of the immediate perToi^^aance Lest, (degree of initial learning) as 
an independent variable enables the experimenter to: a)' determine 
the degtee of ^the relationship betv/een the two repeated measures, 
b) account for effects of individual dilferences between^.p-ax-t icrl-^"^ 
pants in the experiment, and theif^by^, ,cj .jD^bta-in: a ino^re accurate ^ 
estimate of. the o^y.eraH ef-fac^s^iqft tri^^1>^ent variables in tlie ek- » 
:.per i^enTr. ' " \ ^ ^ * ' "'4 % * 

Several, researchers have determined tha»t?^]t '|^.ep^a.ted measures 
design Ts an app^ropriate one for analysis of da^t^ in retention 
studies.. Campbell and Stanley (196-3), Wodtke (1967), and Cook (1970) 
haye giv^n attention to th'fe potential of repeated measures designs, 
• and 'each of these, authors i:ites four strengths^inhererit in 5uch 
d;esigns for retention studies. | 

/ First, repealed measures designs provide control for confound- 
ing that could be exerted by car ry-over, ef f ec ts . Such effecits §fteh 
resul't from the order in v;hic»h observations are obtained in, say, 
factorial -*'esigns/. 

. Second, in retention experiments, differences among individuals 

are often quite large Relative to differenc'es the e;cper imenter is 
.attempting to evalua te.' The repeated measures design provides sta- 
tistical controls, for individual dif f er)aflces ; these control© anielior- 
a-i-e difficulties iboncerned with relative size^ of^ differences exist- 
ing in an ^experiment . 

* T'hird^ — irt liui additive model is realistic in an experiment, it ^- 
is iikely that sequence effects will* int^erfere with the determina- 
tion of differences. , Such interference is alio controlled in the-,^,^ 
repeat6d measares design. 

p 

, A fourth strength cite^^ foV repeated measures designs results 
from the cumulative ef feet's al^the three previously cited strengths. 
Due to tljiis cumulative effecjt^the degree of precision of parameter 
estimates increases mor.e rapidly. than in other designs as the numbers 
of subjects per treatment increase^ When repeated measure^ are us(^. 



Tlte'se four strengths are not: characterlGtlc [of tv;o other de- 
sign's whicb are often employed In retention experiments: counter- 
-IwuLanced designs and analysis of aovariance. Thus, a rt?peated ^ 
measures design would seem to be most appropriate for a'^alysis of 
.data jcesuLting. from impl-emen Nation oTf this- study, - 

Data obtair>.ed from the experiment were sfibjected to both p^ara- 
mjet.tic (analysis^. /of variance) and nonpararaetric (\Jilson Chi-Squarc) 
analyses because: a) there -is cuntinuiag controversy regtarding 
proper cojrrditionp for their respective uses, b) it is liUoly that 
assump ti:ons required for parametric techTTiques were violated ifn 
this^/^tudy, and c) parametric techniques are usually superioi|> to 
nonparame tries in po-wer comparisons far the same sample_,sj-Z-C- ^^ 
the parame'triQ assumptions are mct__ 



Analysis of Data 
Parametric and J^onparamet ric Components 



Pata obtained from the stu^djjr were analyzed using both pai?ametric 
and nopparamet r ic methods^. The rationale for this procedure, presen- 
ted in a previous section, focused upon three issues; (a) the con^ 
tinuing controversy regarding proper conditions for use of parametric 
and nonparametxic technique*iS , ^ (b) ^ the likelihood, in strict interpre- 
tation, that conditions s'aid to be necessary for using parametric ^ 
techniques were! viola ted ^in implementing this study, ar\d (c) thi. 
fact that parametric* techu,ique;5 are usually favo.red over nonparame trie 
techniqiies in pdiwer compai^isons for the same sample size when para- 
metric assumptions are ^met* ' •> 



A thre*e-f actio r model x^yas 
with repeated mea^^sur^JS on the 
follows : 



V ' 



= M +\t, 



employed in 
last facjtor 

1 



the ^parametric analysil 
,It can be stated as 



+ P. + A. 



R- 



Where ; 



RA 



+ TA.,, + PA 



jk 



kl(ij) 



T '= time engaged in instructibn; 2 leVels; fixed* 

P = jquality of instruction; Z, levels;| fixed. * , \ 

A =\ *perf ormance assessment; , 2 levels ;^ immediate and 1 clayed. 

R = r^licatiofts; 30 ^s per treatment^; random. * V 

The two levels of time\engaged in instruction were 75 and 105 min- 
utes. Quality of insCr\UQtion levels corresponded to immediate per- 
formance levels obtained using each of two, versions of the instruc-^ 
tional materj,ais, and tlie two levels of performance ass.essment em- 
ployed were immediate anUMelayed, • 1 f ^ \ 



The nonparametric techn^ique used was a dist.ribution-^f rcc . tes^, 
for analysis of variance hypotheses "devuloped by 'Kilson *(1956) 
Thi% technique -ii/as derived from an c^arlier work hy Rao (1952) which 
demonstrated that a Chi-squlare statistic f^r a contingency tabic can . 
be bro ken* down in much the same way as a total sum of squares is do-* 
^composed in pa^amefcHcJLc *tinalysisi'of -variance com\putations •* 
^fpplicable to problem^ lay.plvln-g designs pf two j^r more vax*iabXe^. 
The general-formula used in this* p.rocedure ist* • 



e: 



.jk "b/l 

I 



n^ = number of observations greater than or, equal"' to tTie overall 
median on the delayed performance asses^siaent , 



A* 



= "number of obseicvations less than the median on the delayed- 
, performance asse^ssment A ' - 

« ' ^ ' ^^t* ' 

afijk number of ofispri^i^tions iu a cell greater thaij or equal to* 
* th^ overdll median on the delayed per formance assessment . 

bfiji^- number^of observation^ in a cell Ifess than the^ overal,!. 
\ m^edian^on the delayed performance asses smetj^L. , ^ • 

ilk ^ -total obs^erva.tlons inj a cell, both itbove and »^qual to, and 
belov;' the*^ ovlSt>all medfian on the delayed perfoi^mance assess- 
* ment . ^/ • ^ , J' 




n 



This technique is not ada:ptable to repealed measrares design/ so the 
auaXagous i^arametric statistical model for the ii^onparametr Ic^analyslj 
is stated in .the form of a three f^jctor factorial design, as follous 

^ijkl = M -f Ti + p. +.A,^ . + TaI^ + PA^j^ + TPA.^j^ + R^i^^j 



Methods of Data Analysla> \ 

Data' resulting from the study were subjected to an analysis of 
variance for the parametric^ analysis using computer program ifSV frofri 
the JUCLA biomedical series (Dixon, 19J0) . Output from the pro^gram 
includes cell means', cell deVia tions,\nd, -Vin anal^^^l s^of variance 
•source table which in<^ludest {sources 6f vartaqce, sum's o£-l^quarcs, 
degrees of fj:eedom, mean squares, and exp^cteff mean squayesV^r terms 
.speciflied in^the model.' The p^irametric ANOVA output for , this stud:/' 
Is presented in' Table 1 and des<:ript^ve data for the experilnci^t Ir^ 
presented fn Table ^2. - N * - • , * * 



. lu [^arforming £ -',tests^ for the parametric analysis, an alph^i* 
i,ev^el *6f^ ,05 was deemed acceptable #with regard to- the probability • 
of committing^ / Type 'I error .in| decisions concerning rejection of > 
the null hypot^heses / » *, • • ' 

, Fof 'the nonpaj;'a^metric' technltjMc, the data were cast in a 
2:x2x2x2* (2 X r k c X b) contingency,^ table . Rows 'in ;the contingency 
tabbies rep.resented thb two levels df quality of instruction , in the 
study.. Coluiivns represented the twV levels of time* engaged in 'in- 
structiop used' in. the ^tud.y.. Blocks reppre's en ted the effects of,* 
, initial learn^ing ind-icated by individuals*' scores on the immediate 
per f t>rmatlce asses^smdjit; . * *^ : . • *^ • ' * 

Sub j ect $ .^ere a&sdgned t^ cells in. the cx)iitingency table , 
according to the .£ojLlov;ing procedure.:*. TJie over'all medians for dis- 
tributj.oijs •of scores on the inj^mecjiate ^and delayed, performance assess- 
rnQTits, r ^spef: titfeiy, wer e 'de'teirmined Subjects with scores equal 
fijo or above the jnedian on the delayed^ performance , assessment were 
u'ssijj.ived *to ojie pOrt,ioh of thje contingency table (n^)., and those 
x^ith,, $cbtes on. 'tiie same measure which were below the overall median' 
^irere assigned to. another portion o.f the* t^able (n^) Depending upon . 
the treatment group they h^id been assigned to, they were subsequent-, 
.^]^y assig'ned^ to a cell fi'n Vk^ appropriate portion oi the table depend- 
ing *uf>pn whether thOrf r scOtres on the immediate performance assess- 

• ikent iizeri* ab-ove or equal to,* or below, tihe overall median on tl^at 
mea^ur^. The results of implementing these cell assignment proce- 
dures for this study^^ ate presenLed in Tabl0 3, " \ 

* In performing' Chi-squafb tests for the rlonparametr ic ^na^lysis, , 
an alpha level ok. .05. was d-eemed accepta.ble with, regard to the proba- 

.bilij;y of. committing .a Type t erVbr in decisions concerning rejection 
.pf; the itull' hypo theses . / 

Table 4 presents results \t the Chl-square tests V7hich v/ere 
perjEormed in th^ course of the ^lionparametr i c analysis. Obtained 

.Chi-squaij'e values aire not presented for discrete* interaction effects 
because the total possible valu^ of Chi-sqtiare attributable to in- 
teraction was pot statist ically ,-s^ignificant for the two-v/ay inter- 
action wi th j^t'Jie least degrees of, freedom. Therefore, there could 

*be no signif iytant intera^Jtion effect fn the npnpara'metr ic analysis. 

Tfic* reader . sha,ul4 t^e aw.are that while each of the two statisti-. . 
cai .analyses used in' the study was .apprdrpr iate , they were not parallel 
The Wilson Cjxi-Square did not allow use of repeated mjsasures, .there- 
fore tl^e remits of the Jtosts should be viewed"" as 'being de*r ived . f r om 
twO" si2p5^rate analyses. ..While the results of implementing the tWQ * 
technique^ will be discussed together in sections that follqv; and 
conclusion^ inferred ^from the two tests are discussed as being re- 
lat^id, there.is no assumption that the tv;o analyses are actually the 
same, " / ' . • 
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' ,.■ Effects of 'Differing, Time" Levels in Xnstructioit , 

. -1 ~\ T— ^vt^-^— : -v, . • . . 

. The .paranvetric nul.l *£6rm of t/h^" s ta t is t ical liypothesis^ 

* tested for this experimental hyp^othesis was : 

ResVlt'^ of the ^ana-lysis of varianc^ were used to perform 
F h Aest at r^lpha = .05,* with 1 end 116 degrees of fre,edom. 
*The r^'jection'region for this' test is represented obtained 
F y,alues greater than 3.93c} ,The obtained F 'f or tl/is hypothe- 
s^iTs was 6.8, therefore,' the njall Hypothesis wa»s r/ejec*ted. 
/The effect of differing time levels in the experiment was,' 
found to b.e significant . Acco-'rding to " information provided 
by Gohen (1969), power for thjl;s statistical te^t yras found to ' 
be aj^proxima.tely .97 f:or a moderate ef f ec t sj-ze 

T4ie npnparame.tric null form, ot the"^ statistical hypothesis' 
tested was: MIof There are no, differences among groups which 
can be attribute^ to effects by diff'erlng time levels in in- 
struction. ' . • ^ , • ! 

. \ Results obtained were tfisted using a Chi-sq^uarc' s tatistlc 

Y 3u^4.th alpha = .\05, and 1 degree of freeciom. , The r ej ec tion regia,n 

* or .this test consistfed of Chi-square values greater than ,3^84. , 
i^he obtained Chi-^squate value for this test was 1.6 (power = 

^ *93 for a .moderate effect size') which did not^allow rejection ^ 
of the^ null hypothesis. Therefore,' the effects of us.ing dif-|y^\ 
fering time levels in the experiment was. fou-rid to b.e not sig-% , 
nif leant, in the nonpararaetr ic "test . • * ' 

' ^ • I ^ ■ " ^ . . ' 

The deoisioh toj^reject the null hypothesis in the parametric 

• case, meant that ' t\\^ alternative hypothesis (na: Ti^PiS 

. could not b\e rejected. This decision supports the cont^en^ioij 

V that dif'fer .ng levels of achievement in delayed letention were 
proraoteci by an increased amount .of study ,time.* Failure to ^ 
reject th^^null hypothesis in the nonpararaetric test disallowed 
any inference with respect to the ^ alternative hypothesis. 

These teato proved to be the only occasion in the expeirimen.t 
.where similar st'ajtistical decisions were not supported Vy , 
s.ults jof bath parametric ^nd nonparamctr i c tests . 'In the non-r 
parathetric/aha^lysis o'f djata, immediate posttest scores were / 
arbitrarily divided at the median qf that distribution^ (see 
- Table 3) jco provide for use of amount of initial learning as 
a variable in the analysis. This decision, while consi.stervt 
with directio.ns for use of thi? particular Chi-square technique 
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(Wilson, 1956J , promoted manifestation of "one issue of the continu- 
ing parame tric'/nonparametr ic controversy in the r<xsult5 of tl)ip.. eX- 
perimenJ. That is, valid parametric tests are more, sensitive . to 
differences and therefore^ are. more powerful for ^he same experimen- 
tal conditions than aire similar noppatame tr ic techniques, when all 

parametric assumptions ar^ met^ ♦ ' \ 

• > ' , . 

In preparing the dapta for the npnparaffletr ic ahalysis, the 
level\of measurement necessarily used was nominal as opposed to the 
inte^fval level used in ANOVA. The particula^r classification syst^em . 
empl^o^jed- in the VJilson Chi-'s^quar e test calls for dividing scores 
into two groups; those below the median '^and those abovq o.r equal to 
t^ie median.. Thus^ differences in ind ividual ^scores , on the orde;r 
of, ^ay, 10 points on an interval s.cale, are not evidenced In the^ 
nominal scale employed in the nonpar ame trie analysis. Tfre reader 
may determine, for himself.., .which of the^^' data scale assumptions 
meets his criteria. '* * ' . 

Ef f ec t\s of Differing Quality Levels in Instruction 

The parametric null forpi of the statistical hypothesis .t^^sted ^ 

was: , =0. An F^-^ test w.a:s_.jLexLQj:.me.d 

Quality, Levels * . ' ^ * 

at'alpha = . 05 "and 1 and' li6 degrees of freedom.. The rejection ^^^--^^ 
region for this test consistency of obtained F values greater t^han^-- r 
3.93. The obtained £ far this test wa's 4..4, which allowe^L reject i<^ * 
of the null hypothesis, and the power level assoclaj^ed with the 
decision was a^pproxima tely ^,97 for a,_moderat^e^'feet size. ' ^ 

_The nonparametr^ic^ sta^tistical l>;ypothe^is tested was: Ho: 
Ther« are rtd differences among gir^oups which can be attributed to 
effects of differing quality of instruction leVels.' The obtained ^ 
Chirsquare value for this hypothesis was 4.4, which allowed rejection 
^of the null hypothesis at a power level of approximately .93 for a. 
moderate effect size. t he re j ec t i5n. region was represented by ob- 
tained values greater titan 3.84-, f o*r 'Chir-squar e with alpha = .05 \ 
and 1 degree of freedom. * ' ' • > > 

Rejr .ing the null hypothesis in the parametric and nonparametr ic 
tests meaiit that the respective? alternative hypotheses 

(Ha: 0^ Quality Levels '- There are differences 

am.ong gro^up^s wnicn C'cWTe 'attributed to effects* of differing quality 
of- instruction levels) could not be rejected, i.e. , the use of 
differing! quality levels of instruction can promote a,chievement of 
differing delayed retention levels. ' 

Effects of Amount of Initial Learning * ' , 
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The parametric statistical hypothesis'/ tested was: 

Tymount. Initial Learning _ ^ The obtained £ value 'for 
the hypottiesUs was 128.4. The rejection region consisted of obtained " 

^' ^ . . ■ 

■ 15 ■ ■ ■ 
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£ values at alpha = .05 with 1 and 116 degr,ees of freedom, greater 
thai) 3*83. K The null hypothesis was. rejected v;ith a power level of 
approximately ^7 -for a moderate effect size. 

• ' , ■ \ 

The nonparametric statistical hypothesis tested (Ho: There are 
no differences among groups -whTch can be attributed to effects of . 
^ amount of initial learning) was rejected with, an obtained CJji-square 
value* of 25.3^. -The rejection region ,for the hypothesis co'nsisted of 
obtained Chi-square values, with alpha = .05 and 1 degree of free- 
^ dom, which v;ere greater than 3.,84> A ppwer level of approximately 
• .93 was- associated with the decisioti to reject tUe null hypot^hesis 
for a moderate effect size.' * 

Rejec^tion of the null hypothesis^ in the parametric test meant 
th^t tlve alternative hypot,h.esls (Ka* *cr^ - * ?^ 0) 

. ^ ' "^Amount^Iniiial Learning- 

could *not be rejected. A' similar conOTtTsxon wdTtirirespec t to the 
nonparametric null hypothesis was. appropriate, therefore, the air- 
tentative hypotheses (Ha: Therje are differences among groups v/hich j 
can bf attributed to effects of amou;nt of initial learning) could j 
not be ^r ej ecp-ed ; i.e. that differences in amoupts of initial learn- j 
ing promote diff.ering levels, of achievement in delayed retention. , 

Effects of Interactlon.<*j Among Variables 

Statistical null hypotheses for all interactions among the 3 ' 
/ indep.endept variables were tested using parametric means. Ka^qbtai^ned 
Y was suf f iciently ^large to allov; rejection of an.y^^^ull hypothe&i^ ^ 
concerning , interactions (see Table 3). .The rejection region for £ . 
"with alpha = .05 and .1 and 116 degrees of freedom c>onsists of F j ^ 
values greater than 3.64. The largest £ for any interactfion hypo-, 
thesis was 7 . The power level associated \^xt\i this, statistical | 
.decision was apiprbxima t eiy .92 for ^ a -moderate effect size. | 

As reported previously, no nonparametric test allowed rejectijon 
of^any statistical null hypothesis concetned^ with interactions,, a^ 
the obtained Chi-square Value for total int^ractio.n (2.2) v/as not 
\signiflcant prior to partitioning it into the tv/a- and three- way/ 
Int^erac tions . .Thi-s negated the possibility that any discrete intpr- 
action coul4 "be found to be signjLf icant by nonparametric methods 
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Limitations 



There are, three possible sources kno^^n that could" exert limnLt- 
ing i,nf luenc.es upon results in the preXenyt study. Two of the so/urces 
will be reported here and one will be r6dorted later in the Unant ici - 
pated Outcomes section of bhis paper. / j 

* » First, there is the possibility that confounding existed^ between 
t|ie variables of quality levels of instyruction, and the amount |o.f 

/ 

.16 ' . ■ , 
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initial learning.. Quality levels of instr^ucticn were obtainis^ far 
Liie tuo vers5.ons of the programmed text by determining tlYfe proportion 
of subjects vjho achieved an arbitrary performance criterion level 
during the formative evaluatipn c.ycle. Therefore, when subjects 
were assigned to treatment groups, especially for the shorter time 
period (75 minutes), only a certain proportion of them (33% for 
version I and 63% for vers.ion II) were likely to achieve the arbi- 
trary performance criterion level. ' . - 

At the same timc^, amount of Initial , l;earning for each 'indivi- 
dual, in the experiment \^as measured by t,he riumber of items correc^t 
on the immediate posttest. 'It .is possible , .kssuming the logic of 
the Carroll model to be cprrcct, that.^the combination of quality 
level of instruction and time allotment employed in each treatment 
group .xiOTis titu ted an up.per limit on the numt^er of students who co.uld 
ac.hJLeve a* degree of ini^ial learning equal :tp the arbitrary minimum 
performance level. That is, for a given quality level of instruc- 
tion and ai^y time allotment that is le-ss than what is needed for 
each individual in a group, it is only possible for some proportion 
of the group less than 100% to achieve a desired per formance, level . 
Confounding x^ould result in that we may not have accurately deter- 
mine^ relative effects, upon the immedjiate performance level achieved 
by an individual (as contrasted to^ the average student) 'of time 
allotment versu s quality level of instruction. 

The s.econd possible source of limitations on the results pf* 
this scurd7 concerned a promotion being operat^jd by a local radio 
station. Under the rules of the promotion, the school which turned 
in the most 3x5 .inch aards with the words "Care ?ree Gum" written 
on Chem would win a co-noert *t the school by an Internationally 
known rock music group. The two days during v/hich the programmed 
texts and immediate performance tests were implemented were also the 
final two days of the contest and subjects were quite interest'ed in 
writing cards. While subjects did not engage in producing cards 
when they were worklrig in the two versions of the programmed text, 
it was possible that they were not attending fully to the task.' 

Thirteen days later, when the experimenter retutned at t.he 
appointed time to admin;Lster the retention test, the students were 
not in the large lecture classroom. An inqiuiry at the school office 
led to /the discovery tliat the subjects were in the auditorium where 
they \/er.e being entertained by an internationally know rock group-- 
a prize won by the school in the "Care Free Gum" contest. About 
twenty minu tes later the sub j ec ts returned to ^ take the retention 
test. It is possible t4ia t the "Care Free G.um" phenomenon '^xeirtcd 
effects upon the r^e suits of this study to some, unknown ex ten t . 



/ TABLE 1 
A Analysis of' Vari/^ce i 

/ ^ 



Source., 

4 * 


d.f . 


MS 


F 


^ Mean (M)^, 


1 


75650 




Time engaged in instruction (T) 


1 


424 


6.8* 


Quality of instruction (P) 


1 


i 275 . 


4.4* 


Immediate performance assessment (A) 

♦ / , 


I 


1799 


128.4* 


. T X ,P 


\ 1 


18 


.3 


' T X A' • . *' * 


';l 


' 4- 


.3 


■ . P X A 
^ T X. P- X A 


1 


26 
6 


1.7 
.4 


' ^Subjects (S) within treatments 


116 


62 




S X A within treatirients 


■ 116 


15^ 




* .significant with* alpha = .05 


^ 





.TABLE 2 

Descriptive Data for' Experiment 



Group 


Test 


Rahge 


Mean 


'7 

standard Deviation 


version I 
75 minutes 


posttest 


24 


18. M 


7.65 


delayed retention^ 


.20 ' 


12.7 


\ ' ■7.35 


version II 
75 minutes 


posttest 


26 


20V8 


■ ' 5.5-8 


delayed retention 


14 


15 -jg. 


• 4.60 


version I 
105 minutes 


posttest 


25 


, 22.4 


■5.70 


delayed retention 


24 


15.4- 


5.90 . 


'Version II . 
;105 minutes 


posttest 


26 


2 6.; 4 


6.85 , 


delayed retention 


22 


17.8 


6.65 



TABLE 3 

Nonparametric Contingency Table 



(above overall 
Md on delayed 
assessment) 



Quality of 
Instruction 



Level 1 
(Lqw) 



Level. 2 
(High) 



Time Cngaqeci in Instruction 
7 5 minutes' ' 105 minutes 



<Md , 
immediate 



3 



•7 



>Md 
immediate 



6 
9 



immediate 



>Md' 
immediate 



12 



16. 



(below overall 
Md on delayed 
assessment)/^ 

Quality, of 
Instruction 



/Level 1 
(Low) 



Level 2 
(High) 



13 
10 



11 



10 



TABLE 4 

Results of Nonparametric Analysis 



Source 


df 


Chi-square 


Total Chi-square , 


7 


33.5* ' 


Time engaged in instruction (T; column Effect) 


■ I, 


1.6 


^Quality of Instruction (P; row effect) 


1 


4.4* 


Immediate performance assessment (A; block 


1 


25.3* 


r effect) 






Interaction effects (total). 




2.2 


* isignificant with alpha = ,05 ^ 7 

»^ 1 


-4) 
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